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Abstract.  This  paper  presents  a  solution  to  certain  problems  in  switched 
controller  design  for  stochastic  dynamical  systems  with  a  quadratic  cost. 
The  main  result  is  a  separation  theorem  for  partial  information  systems. 
This  result  is  then  used  to  convert  the  partial  information  stochastic 
control  problem  to  a  complete  information  stochastic  control  problem. 
We  also  show  that  certainty  equivalence  does  not  hold.  The  optimal  se¬ 
quence  of  controllers  can  be  determined  via  an  appropriate  solution  to  a 
dynamic  programming  problem. 


1  Introduction 

There  are  many  real  industrial  control  problems  where  the  control  action  is 
determined  by  switching  among  a  finite  number  of  given  control  laws.  Typi¬ 
cal  examples  include  an  automobile  transmission,  where  different  gears  need  to 
be  switched  in  order  to  obtain  desirable  performance  (gain  switching)  [1,2],  a 
thermostat  where  the  plant  heats  up  when  the  control  is  on  and  cools  when  the 
control  is  off,  and  steering  an  automobile  where  switching  between  appropriately 
chosen  controls  laws  can  make  an  otherwise  uncontrollable  vehicle  controllable 
and  allow  the  vehicle  to  be  steered  (locally)  in  any  desired  direction  [3](pg  76). 

In  [4,5]  we  devised  optimal  strategies  for  switching  between  given  controllers 
at  fixed  switching  intervals  for  plants  described  by  an  uncertain  differential  equa¬ 
tion.  In  this  paper  we  consider  optimal  switching  strategies  for  plants  described 
by  linear  time-varying  stochastic  differential  equations  with  a  quadratic  cost 

*  This  work  was  supported  by  the  Center  for  Sensor  Signal  and  Information  Processing 
the  Australian  Research  Council  and  by  US  Army  Research  Office  under  the  MURI 
grant  “An  Integrated  Approach  to  Intelligent  Systems”,  grant  D A AH04-96- 1-0341. 


functional  and  develop  optimal  switching  strategies  for  both  the  full  and  partial 
information  cases  [6,7].  We  consider  separation  and  certainty  equivalence  for 
switched  systems.  Although  these  notions  have  been  studied  for  many  years  [8] 
and  their  applicability  to  various  classes  of  stochastic  systems  continues  to  be  of 
great  interest  e.g.  [9].  To  our  knowledge  these  important  notions  have  not  been 
investigated  for  linear  switched  controller  and  hybrid  control  systems.  This  pa¬ 
per  presumes  a  background  knowledge  of  linear  stochastic  control  systems  and 
Ito  calculus.  An  excellent  treatment  of  these  subjects  can  be  found  in  [10]  and 

[ii]. 

In  this  paper  we  show  that  the  full  information  problem  with  quadratic  cost 
the  optimal  switching  sequence  of  controllers  is  given  by  a  solution  to  a  dynamic 
programming  equation.  We  show  that  where  the  switching  function  depends  on 
both  time  and  state  (state  estimate)  certainty  equivalence  does  not  hold.  For 
the  partial  information  case  a  separation  theorem  is  established.  This  is  then 
used  to  convert  the  partial  information  problem  to  a  full  information  problem 
enabling  the  optimal  controller  switching  sequence  to  be  determined. 

The  organization  of  this  paper  is  as  follows.  Section  2  considers  full  infor¬ 
mation  switched  controller  systems.  The  main  results  of  this  section  is  to  show 
equivalence  between  the  optimal  controller  switching  sequence  and  the  solution 
to  a  dynamic  programming  equation.  In  this  section  it  is  also  shown  that  for  the 
class  of  systems  considered  in  this  paper  certainty  equivalence  does  not  hold. 
In  section  3  the  partial  information  problem  is  considered.  The  main  results  of 
this  section  are  given  in  terms  of  the  existence  of  suitable  solutions  to  a  Ricatti 
differential  equation  of  the  Kalman  Filtering  type  and  appropriate  solutions  to  a 
dynamic  programming  equation.  In  section  4  a  simulation  example  is  presented. 

2  Optimal  Controller  Switching  with  Full  Information 

All  random  variable  in  this  paper  are  defined  on  the  probability  space  (12,  A,  V ). 
Let  it>(*)  be  a  vector  valued  Wiener  process  with  covariance  matrix  Q(-).  Let  £ 
denote  expectation  with  respect  to  the  measure  V.  Let  N  E  N+  be  a  given  pos¬ 
itive  integer.  The  permissible  switching  times  {toAiA2y  *  An}  corresponding 
to  an  ordered  discrete  set  where  (f,-  6  R+). 

Consider  the  following  stochastic  system  whose  dynamics  on  the  interval 
[0  An)  are  given  by  the  stochastic  differential  equation 

dx(t)  =  (A(t)x(t)  +  B2(t)u(t))dt  +  Bx(t)dw(t)]  x(0)  =  xo  (1) 

where  x(t)  E  Rn  denotes  the  state,  u(t)  E  R*  is  the  control  and  the  functions 
A(t),  B\(t ),  B2{t ),  C(t)  are  given  piecewise  continuous  matrix  functions  of  time 
of  appropriate  dimension. 

Controller  Switching  with  State  Feedback  [12,5,4,13].  Given  a  finite  collection 
of  controllers 


t*i (t)  =  I7i(<,  x(t)),  u2(t)  =  U2(t,  x(t)),  . . . ,  uk(t)  =  Uk{t ,  x(t))  (2) 


where  the  control  laws  J7i(*,  •),  f72(*,  C4(*>  *)  are  given  continuous  matrix 

functions.  Let  Ij(-)  denote  a  state  switching  function  which  is  permitted  to 
switch  between  controllers  at  set  switching  times  tj,  {j  =  1, . « . ,  i\T}.  The  class  of 
switching  functions  considered  in  this  section  is  of  the  form  Ij(-)  :  {x(-)  |qj}  — > 
{1,2,. fc},  which  map  from  the  set  of  state  measurements  (ar(-)  |qj}  to  the  set 
of  symbols  {1, 2, . . .,  Ar}  which  index  the  controllers  (2).  This  class  of  switching 
functions  defines  a  family  of  dynamic  nonlinear  state  feedback  controllers  of  the 
form: 


Vi  G  {o,  u(t)  =  E^. (*,*(*))  Vf  €  [tj,t(j+ 1))  where  ij  =  /,(*(•)  fr). 

(3) 

Hence  the  control  strategy,  which  is  a  rule  for  switching  from  one  controller 
to  another,  constructs  a  symbolic  sequence  from  the  set  of  state  mea¬ 

surements  ar(-)  |oJ.  The  set  of  all  admissible  controls  achievable  by  controller 
switching  (3)  with  the  controllers  (2)  is  denoted  by  U . 

Let  S(-)  be  a  given  function  which  maps  from  Rn  to  R,  let  N(t)  and  M(t) 
be  given  matrices  of  time  satisfying  N(t) ,  M(t )  >  pi,  p  >  0  Vf  £  [0,  fjv],  Xf  >0 
is  a  given  constant  matrix  which  penalises  the  final  state. 

Introduce  the  following  quadratic  cost  function, 

W(t,x,u(t))  =  £  jar  (t)N(t)x(t)  +  u  (£)M(£)u(<)j  (4) 

and  define 


F}(x0 ,5(0)  =  £[S(x(tj+1))]  +  /  W&x^tydL 


rtj+i 

Jtj 


(5) 


Definition  1  fState  Feedback  Stochastic  Control  Problem ),  The  state  feedback 
stochastic  control  problem  with  system  (1)  is  said  to  have  a  solution  via  controller 
switching  with  the  controllers  (2)  if  the  following  conditions  hold: 

(i)  For  any  admissible  control  sequence,  u  £  U,  there  exists  a  unique  trajectory 
of  system  (1)  on  the  interval  [0,tfjv). 

(ii)  The  cost 


J{u)ueu  —  £  ^ J  * 


x  (t)N(t)x(t)  +  u  (t)M(t)u(t)dt  +  xtNXfXtN  J  (6) 


) 


exists  and  is  finite  for  some  admissible  control  sequence,  u  £  U. 


Theorem  1.  The  state  feedback  stochastic  control  problem  (Definition  (1))  for 
system  (1)  has  a  solution  via  controller  switching  with  the  state  feedback  con¬ 
trollers  (2)  if  and  only  if  the  dynamic  programming  problem 

Vn(x0)  =  £  [x'tNXfxtN] ;  Vj(x o)  =  .  min/  [F}(x0, 1$+1(.))]  (7) 

has  a  solution  for  all  j  =  0, 1, . . N  —  1  and  all  Xq  £  Rn. 


Furthermore  let  ij{x o)  be  an  index  such  that  the  minimum  in  (7)  is  achieved 
fori  =  ij(xo).  Then  the  controller  (2),  (S)  associated  with  the  switching  sequence 

where  ij  =  ij(x(tj))  solves  the  state  feedback  stochastic  control  problem . 
Proof  Sufficiency.  For  any  j  =  0, 1, . . . ,  N  introduce  the  cost  to  go 

Vj  ~  Jt)ZU£  (jt  *  +  u{t)M{t)u(t)dt  +  xtNXjxt)j  .  (8) 

Following  [14, 10]  Vj  (•)  satisfies  (7) .  Also  it  follows  from  (8) ,  that  if  the  stochastic 
control  problem  has  a  solution,  then  Vo(xo)  exists  and  is  finite. 

Necessity.  Equation  (7)  implies  that  for  the  controller  associated  with  the 
switching  sequence  {ij } we  have 

£{J0  x(t)N(t)x(t)  +u(t)M(t)u(t)dt  +  x't„Xjxt„SJ  <  Vo(z(0)).  (9) 

Since  \^(x(0))  =  J(tt*,x(0))  <  Jueu{u,  x(0)),  the  controller  associated  with 
the  switching  sequence,  solves  the  state  feedback  stochastic  control 

problem. 

Suppose  that  the  given  state  feedback  controllers  (2)  are  of  the  form 

«i(0  =  u2(t)  =  K2(t)x(t),  uk(t)  =  Kk{t)x(t)  (10) 

where  AT,*(f)  are  given  matrices  of  time,  then  we  have. 

Definition  2  Certainty  Equivalence  [10,  6].  The  system  (1)  is  said  to  satisfy  a 
certainty  equivalence  principle  if  the  optimal  admissible  control  law  is  indepen¬ 
dent  of  the  statistics  of  the  noise  inputs . 

Corollary  1  Consider  system  (1)  and  state  feedback  controllers  (10).  For  this 
class  of  stochastic  control  problems  certainty  equivalence  does  not  hold. 

Proof  Consider  system  (1)  with  the  state  feedback  basic  controllers  (10).  All 
that  is  required  to  be  shown  is  that  the  controller  switching  sequence  for  the 
plant  with  no  noise  is  different  to  that  with  noise. 

The  cost  J(u)  for  all  admissible  controllers  can  be  rewritten  as 

«W«)  =  £  (*tNX,xtN  +  fj  x'(t)(N(t)  +  K'itR(t)Kit)x(t)d)  .  (11) 

Expressing  the  state  as 

i(<)ii(<)  +  i*(<)  (12) 

with 

x(t)  =  (A  +  B2Kit)x(t),  x(0)  =  z(0)  (13) 

dx*(t)  =  (A  +  B2Kit)x*{t)dt  +  dw(t )  x* (0)  =  0,  (14) 


it  follows  that 


€z*(t)  =  0,  Sx  (t)x*{t)  =  s'(t)£s*(t)  =  0  V*.  (15) 

Following  standard  dynamic  programming  we  substitute  of  (14)  and  (15)  into 
(11)  and  rewriting  (11)  as: 


Jueu  =  £  (x'tNXfxtN)  +  €  (^J*N  x(t)(N(t)  +  KitR(i)Ki,)x(t)dt'j  (16) 
+£  (^J*N  x'\t)(N(t)  +  K'itR(t)Kit)x*(t)d?J 

)x(t)dt'j  . 


n= 0  Jtn 


x(t)(N(t)  +  KitR(t)Ki,)a 


It  can  now  be  seen  that  in  general  the  choice  of  gain  matrix  that  optimizes  the 
second  and  third  terms  in  the  above  expression  (16)  will  depend  on  both  the 
state  as  well  as  the  noise  covariance  and  its  contribution  to  the  state  over  the 
interval  [fjv-ijtjv)-  This  contradicts  certainty  equivalence. 


Remark  In  switched  controller  systems  the  optimal  controller  switching  sequence 
for  linear  feedback  controllers  depends  on  the  input  noise  covariance.  This  result, 
also  applies  to  zero-mean  white  Gaussian  process  noise  inputs.  This  differs  from 
optimal  linear  quadratic  Gaussian  control  [7,6,15]  were  the  optimal  feedback 
gain  is  independent  of  the  input  noise  covariance. 

We  illustrate  that  Corollary  (1)  above  by  considering  the  system  (1)  with 
parameters  (29)  in  the  simple  scenario  corresponding  to  only  one  switching  time 
with  switching  occurring  at  T  =  0.  Let  the  system  be  defined  over  the  interval 
[0,0.1)  with  Xf  =  0.  The  process  noise  w(t)  is  zero  mean  Gaussian  with  covari¬ 
ance  £(w(t)w  (t))  =  1.0.  For  this  plant  the  cost  (when  there  is  no  process  noise 
is  given  by  the  second  component  of  (16))  and  is  equal  to  J(u\)  =  x^Z\x^  for 
controller  u\  and  J(«2)  =  xo^2^o  for  controller  U2  where 

0.5701  -0.0873' 

-0.0873  0.3459  * 

When  there  is  noise  the  cost  of  using  any  of  the  given  controllers  is  determined  by 
the  second  and  third  components  of  (16)  and  is  equal  to  J(ui)  =  xfQZiXQ-{- 0.0152 
or  J(u2)  =  Xq^2®o+0.0210.  Consider  the  point  x  =  [0.1, 0.1]  for  the  case  without 
any  noise,  «2  is  the  optimal  control  to  use,  whereas  with  noise  the  optimal  control 
to  use  is  wi¬ 
lt  can  be  clearly  seen  from  Figures  (1)  and  (2)  that  the  controller  switching 
regions  depend  on  the  process  noise  covariance.  This  differs  from  standard  L.Q.G. 
optimal  control  were  the  controller  gain  matrix  is  independent  of  the  input  noise 
covariance  for  zero-mean  Gaussian  process  noise. 


[0.5159  -0.0314] 
[-0.0314  0.3257  J 


3  Optimal  Controller  Switching  with  Partial  Information 

Let  £  be  a  random  variable  with  Gaussian  probability  law,  mean  xq  and  co- 
variance  Po  and  u(-)  is  a  Wiener  processes  independent  of  tu(-)  with  covariance 
matrix  P(-).  Consider  the  following  stochastic  system  whose  dynamics  on  the 
interval  [0,t/v)  are  described  by  the  stochastic  differential  equations 

dx(t)  =  (A(f)x(t)  +  B2(t)u(t))dt  +  Bi(t)dw(t),  x(0)  =  £, 

dy(t)  =  C(t)x(t)dt  +  dv(t),  (17) 

where  x(t)  £  Rn  denotes  the  state,  y(t)  £  Rp  denotes  the  output  and  u(£)  £  Rh 
is  the  control  input.  The  functions  A(t ),  Bi(t)}  B2{t)>  C(t)  are  known  given 
piecewise  continuous  matrix  functions  of  time.  We  assume  that  P(t)  >  pi,  p  > 
0  WE  [0,  tjv),  and  we  define  the  following  two  families  of  sigma  algebras 

F*  =  <r(£,w(s),y(s),s  <t) 

Z*  =  <r(y{s),s  <t)  (18) 

Controller  Switching  with  Output  Feedback  [12,5,4,13].  Given  a  collection  of 
output  feedback  controllers 

t*i(*)  =  Ki{t)y{t),  u2(t)  =  K2{t)y{t),  ...,  uk{t)  =  Kk(t)y{t)  (19) 

where  K2{‘)) . . Kk(-)  are  given  continuous  matrix  functions  of  time,  we 

define  a  switching  function  &j(-)  which  is  permitted  to  switch  the  controllers  (19) 
at  switching  times  tj  {j  =  1, . . N}.  The  class  of  output  switching  functions 

considered  in  this  section  is  of  the  form  ^j(-)  :  {Ztj}  — >*  {1,2,  which 

map  from  the  output  sigma  algebra  {Ztj}  to  the  set  of  symbols  {1,2 
which  index  controllers  (19).  This  class  of  switching  functions  defines  the  family 
of  dynamic  nonlinear  output  feedback  controllers  of  the  form: 

Vi  €  {0,1,... ,1V- 1}  u(f)  =  /£!,.(<),  y(<)  Vt€[tj,tj+1)  where  ij  - 

(20) 

The  control  strategy  now  becomes  a  rule  for  switching  from  one  controller  to 
another  to  construct  a  symbolic  sequence  {t/}^)1  from  the  sigma  algebra  Ztj . 
Furthermore  let  X  denote  the  set  of  all  controls  achievable  by  controller  switching 
(20)  with  the  controllers  (19)  adapted  to  Zx , 

The  solution  to  the  partial  information  stochastic  control  problem  considered 
in  this  section  involves  the  following  Ricatti  differential  equation 

P(t)  =  P(t)A'(t)  +  A(t)P(t)  -  P(i)C'(f)ii(i)-1(<)C(f)P(<)  +  B2(t)Q{t)B'2(t) 
where  P(0)  =  P0  (21) 

We  consider  the  estimator  equations  of  the  form 

x(t)  =  A(t)x  -f  P{t)C  (t)P(t)"'1(y(t)  —  C(t)x)  where  x(0)  =  f  (22) 

which  correspond  to  the  continuous  time  Kalman  filter  [16]. 


Let  S(-)  be  a  given  function  which  maps  from  Rn  to  R  and  let  x0  G  Rn  be  a 
given  vector.  Introduce  the  following  cost  function: 

Wi (*,£,«(*))  =  £  (x'(t)N(t)x{t)  +  u'(t)M(t)u(f))  .  (23) 

Then 

Fj(x0,S(-))±£(S(x(tj+1)))+  fH,+1)  W1(t,x,Ui(t)Mt))dt.  (24) 

JtS 

Definition3  The  output  feedback  stochastic  control  problem  with  system  (17) 
is  said  to  have  a  solution  via  controller  switching  with  the  controllers  (19)  if  the 
following  conditions  hold: 

(i)  For  any  admissible  control  sequence }  u  £  X,  and  initial  condition  x(0)  there 
exists  a  unique  trajectory  to  the  system  state  on  the  interval  [0,  ^isr)  and  the 
solution  to  the  Ricatti  differential  equation  (21)  exists  and  is  positive  definite 
for  all  t  E  [(Mat). 

(ii)  The  cost  given  by 


J(u)uex  =  £  (/  x  (t)N(t)x(t)  +  u  (t)M(t)u(t)dt  +  xtNXfXtNSj  (25) 


exists  and  is  finite  for  some  admissible  control  sequence ,  u  E  X, 


Theorem  2 •  The  output  feedback  stochastic  control  problem  (Definition  (3)) 
has  a  solution  via  controller  switching  with  the  output  feedback  controllers  (19) 
if  and  only  if  the  solution  P(-)  to  the  Ricatti  equation  (21)  with  initial  condition 
P{ 0)  =  Po  defined  and  positive  definite  on  the  interval  [0,tjv)  and  the  dynamic 
programming  equation 

VN(xtN)  =  S(x'tNXfxtN);  Vj(xtj)=  min  €  (Fj(xtj,VH1(xtj+l)))  (26) 

t  — 

has  a  solution  for  j  =  0, 1, . . N  —  1  for  all  x0  E  Rn. 

Furthermore  let  ij(xQ)  be  an  index  such  that  the  minimum  in  (26)  is  achieved 
fori  =  ij(x o)  and  x(-)  be  the  solution  to  the  equation  (22)  with  initial  condition 
x(0)  =  xq.  Then  the  controller  (19)}  (20)  associated  with  the  switching  sequence 

{*j}^oX  where  ij  =  4>j(x(tj))  solves  the  output  feedback  control  problem . 

Proof.  ( Outline )  By  virtue  of  Girsanov’s  theorem,  Lemmata  (1,2, 3, 4)  in  Section 
6  below  and  Lemma  (2.4.3)  from  [6]  it  follows  that  the  evolution  of  the  state 
of  (17)  with  u(£)  =  0,  denoted  a(t)  satisfies  Su[a(t)\Zt]  =  £u[a(t) \J*\.  Thus 
the  states  estimates  generated  by  a  Kalman  filter  are  optimal  when  there  is 
controller  switching  as  considered  in  this  paper.  This  is  established  in  detail  in 
section  6.  It  then  follows  that  under  the  new  probability  measure  Vu  (defined  in 

section  6),  the  estimation  error,  e  =  x(t)  —  x(t)  is  a  Gaussian  process  with  zero 


mean  and  covariance  matrix  given  P(t)  by  (21).  Hence,  if  the  output  feedback 
stochastic  control  problem  has  a  solution  via  controller  switching  it  follows  that 
the  solution,  P(-),  to  the  Ricatti  differential  equation  (21)  with  initial  condition, 
P( 0)  =  Pq,  is  defined  and  positive  definite  on  [Jo>//v]*  Now  from  Lemma  (4)  we 
have  that  the  minimization  of  cost  (25)  is  equivalent  to  minimizing  (26)  where 
x  is  the  estimator  state,  V^x^jv)  =  £ux(t^)Xfx(tjsf)  is  the  cost  at  the  final 
position  and 

dx(t)  =  (j4(*)x  -f  B2(t)u(t))dt  +  P(t)Cl  (t)R(ty1(y(t)  -  C(t)x)dt 


=  (-A(t)i  +  B2(t)u(t))dt  +  dv(t) 

(27) 

where 

£>(<)]  =  o 

€u[i /(*)!/(*)]  =  P  {t)c' (QRity'CMPit). 

(28) 

The  statement  of  the  Theorem  now  follows  immediately  from  Theorem  1. 

Remark  Results  for  discrete  time  switched  controller  systems  can  be  shown  to 
trivially  follow  from  the  results  presented  in  this  paper. 


4  Illustrative  Example 


We  consider  the  2  dimensional,  unstable,  non-minimum  phase  system 

roi 


A(t)  = 


0  1 

-1.25  1 


,Bi«  = 


,B2(t)  = 


;],c«)=[-21' 


(29) 


Consider  the  system  (17)  the  following  set  of  output  feedback  controllers 

«!(<)  =  [-l,2]£(t),  u2(t)  =  [3,  -6]x(<),  (30) 


and  permissible  controller  switching  at  time  instants  jT ,  j  =  0, 1, ,  250,  T  = 
0.05  over  the  time  interval  [0,12.5)  seconds.  It  can  be  easily  verified  that  each  of 
the  controllers  in  (30)  is  independently  unable  to  stabilize  the  plant. 

For  simulation  purposes  set  to  Xj  —  10/  and  the  matrix  observer  Ricatti 
equation  (21)  initial  condition  P(0)  =  /.  The  process  and  observation  noises  were 
also  set  to  £(w(t))  =  0,  £(w(t)w‘ (t))  =  0.1,  £{v{t))  =  0  and  £(v(t)i/' (t))  =  0.1. 
The  matrices  N(t)  and  M(t)  are  N(t)  =  0.1  and 

MW=[o2]-  PH 

Using  the  results  of  Theorem  2  we  see  in  Figure  (3)  that  by  switching  the  two 
unstable  controllers  the  system  state  remains  within  an  a  small  neighborhood 
about  the  origin.  The  control  input  to  the  system,  is  shown  in  Figure  (4).  The 
switching  between  the  two  controllers  is  evident. 


5  Conclusions 


In  this  paper  we  presented  a  solution  to  certain  problems  in  switched  controller 
design  for  stochastic  dynamical  systems.  The  main  result  is  a  separation  theorem 
for  partial  information  systems  which  can  be  used  to  convert  the  partial  infor¬ 
mation  stochastic  control  problem  to  a  complete  information  stochastic  control 
problem.  We  also  show  that  certainty  equivalence  does  not  hold  and  for  different 
noise  statistics  the  optimal  switching  sequence  changes.  Finally  we  show  that 
the  optimal  sequence  of  controllers  can  be  determined  by  solving  a  dynamic  pro¬ 
gramming  equation.  The  results  presented  in  this  paper  can  be  extended  to  the 
infinite  time  interval  after  some  technical  issues  are  taken  into  consideration. 


6  Appendix 


There  is  an  intrinsic  difficulty  in  output  feedback  stochastic  control  problems 
arising  from  the  fact  that  the  controller  and  hence  the  control  depends  on  the 
observation.  This  difficulty  has  been  solved  for  linear  quadratic  Gaussian  stochas¬ 
tic  control  problem  but  it  is  not  immediately  clear  that  these  results  cover  the 
class  of  switching  systems  considered  in  this  paper.  Thus  in  order  to  establish 
Theorem  (2),  we  need  to  establish  the  existence  of  a  new  probability  measure 
in  which  the  output  and  all  the  admissible  controls  are  independent.  This  then 
overcomes  the  difficulty  described  above.  In  order  to  establish  the  existence  of 
such  a  measure  we  will  need  the  following  definitions  and  Lemmata.  The  argu¬ 
ment  here  essentially  follows  [6]  except  that  special  attention  must  be  given  to 
the  switching  times  were  solutions  to  the  equations  describing  the  dynamics  are 
not  well  defined. 

Define  the  processes  a(-),  /?(•)  by 


da  —  A(t)adt  +  dw(t ),  a(Q)  =  f 
d(d  —  C(t)fidt  +  /3(0)  =  0 


also  define  xi,  y\  by 


(32) 

(33) 


dx i  =  ( A(t)x\  +  B2(t)u(t))dt,  xi(0)  =  0 
dyi  =  C(t)xidt,  yi(0)  =  0. 

For  any  admissible  control  «(<)  we  define 


(34) 

(35) 


x(t)  =  a(t)  +  xi(t) 

y(t)  =  a{t)  +  y1(t).  (36) 

We  then  consider  the  following  two  processes  [6] 

ij°(t)  =  exp  ^ j  x'(t)C"(<)i?(t)-1dy-  |  J  x(t),C(<)'fi(<)-1C(t)x(t)dsj(37) 


where  the  superscript  u  is  to  show  the  dependence  of  rf  on  the  the  control  u(t). 

The  process  r)u  (38)  satisfies  the  stochastic  differential  equation  for  all  time 
t  €  [0,  tjv)  ^  tj  where  tj  are  the  controller  switching  times 

drf^r)u{x,Ct)R-1dy,  0)  =  1.  (39) 

Furthermore  we  define  J*  (u)  as 

■/*(«())=  (40) 

£  [»?“(<jv)  (x' (tjf)Xfx(ts)  +  J  i}u(t)z  (t)N{t)x(t)  +  u{t)' M{t)u(t)j  <ft]  . 

Since  the  controls  are  piecewise  linear  (affine  in  the  state)  between  the  switching 
times  on  the  interval  [tj,  tj+i)  all  that  is  required  to  be  shown  for  the  existence  of 
a  new  probability  measure  Vu  which  makes  the  observation  process  independent 
of  the  control  are  that  the  cost,  J*(u(-))  under  the  new  measure  is  finite  and 
that  the  Radon-Nikodym  derivative  [11] 

dVu 

~jp=Vu(tN)  (41) 

is  a  martingale.  This  is  established  in  Lemmata  (1,2,3)  below  which  closely 
follow  [6]  but  with  modifications  required  to  account  for  controller  switching  on 
a  countable  set. 

Lemma  1.  There  exists  a  constant,  C\ ,  independent  of  the  control  such  that 

£»?“(*)  I  a(t)  |4<  Ci  (42) 


Proof  Consider  the  following  approximation  of  rf , 

jj“(<)  =  exp  [jf  x'C'(<)fl(<)_1rfj/-  i  jf  (x'C',(<)ii(*)_1C'(f)xecfe| 

where 

*  (t)  =  «(*) 
eU  (l  +  e|*(«)P)i/>\ 

There  exists  a  subsequence,  denoted  by  c,  such  that 

almost  surely  (a.s.)  rfe  (t)  -¥  r)u(t)  Vt. 

It  follows  that 


(43) 


(44) 


(45) 


d\a(t)\4  =  4|a(<)|2a/(f)  [A(t)a(t)dt  +  dw]  -f  2  [a(t)2trQ(t)  +  2 afQ(t)a]  dt (46) 


drit (OK*)!4  =  4>?“(if)laW|2Q!,(*)  [A(f)a(<)di  +  dw]  +  T)t(t)\a(t)\4x'eC'R  ldy 
+  2t; “(f)  [a(t)2trQ(t)  +  2a'Q{t)a]  dt  Vt  €  [0 ,tN)  #  t,.  (47) 

Upon  integrating  between  (0 ,<),  using  the  smoothing  property  of  integrals  and 
taking  the  expectation  it  follows  that 

*£  Wl*(<)|4  =  m4  +  45  f  n«(s)\a(s)\2a'(s)A(t)a(t)dt 

Jo 

+  2 £  f  T)e(s)  [\a(s)\2trQ  +  2a'Qa]  ds.  (48) 

do 

Using  the  fact  that  £77“  (s)  =  1  Vs  it  can  be  shown  that 

|4  <  5|e|4  +  C-2  (l  +  5  jf\“(S)|a(S)|4<i5)  (49) 

where  C2  is  a  constant  independent  of  c  and  u(*).  Using  the  Gronwall  Lemma 
one  immediately  obtains 

W)M*)I4  <  Ci.  (50) 

Hence  taking  the  limit  as  c  -4  0,  the  claim  of  the  lemma  immediately  follows. 

Lemma  2.  For  any  admissible  control,  u  £  X ,  and  for  all  trajectories  the  cost 
denoted  by  J*(u(-))  is  finite . 

Proof.  It  follows  from  the  Gronwall  Lemma  that 


1*1  (01  <  Cs 


(51) 


Using  the  definition  of  (36)  and  from  Lemma  (1)  it  follows  that  on  the  finite 
interval  [0,tjv)  there  exists  a  constant  Ci  such  that 


*£WI«Wla<ci. 


(52) 


substituting  into  J*(u(-))  the  claim  of  the  lemma  is  established. 
Lemma  3.  rju (t)  is  a  7*  martingale. 


Proof.  From  Lemma  2.4.2  (pg  40)  of  [6]  all  that  needs  to  be  established  is  that 


£ 


f  r]u(t)\u(t)\2dt  <  00. 
Jo 


(53) 


Using  the  fact  that  the  system  is  piecewise  linear  and  f}u(t)  is  finite  for  all 
l  E  [0,  Jjv)  it  follows  trivially  that  there  exists  a  constant  C3  such  that 


The  following  Lemma  is  also  needed  in  the  proof  of  Theorem  (2). 

Lemma  4.  Suppose  that  the  solution  P(-)  to  the  Ricatti  differential  equation 
(21)  with  initial  condition  P( 0)  =  Pq  is  defined  and  positive  definite  on  the 
interval  [0,tjv)  then 

_jnin^Su  (a:  (tN)X/x(tw)  +  J  x  (t)N(t)x(t)  +  u'(t)M(t)u(t)dt^ 

—  iJf1inN£U  (*  (^N)Xjx(tff)  +  J  x  (t)N(t)x(t)  +  u'(t)M(t)u{t)dt ) 

+SU  (e‘(tN)Xfe(tN)  +  JJ  e  (t)N(t)e{t)dt'j  .  (55) 

Proof.  Rearranging  (25)  we  have 

Su  (su  [*'(**)*,*(**)  +  J*”  x(t)N(t)x(t)  +  u'{t)M{t)u{t)dt  |  Z*])  = 

eu  (eu  [(*(**)  -  x(tN))' Xf(x(tN)  -  x(tN))  +  (x(tN)  -  x(tN))'x}x(tN)  I  Z*]) 
+SU  (£u  [x'(tN)Xf(x{tN)  -  x(NT))  +  x'(tN)X}x(tN)  |  Z*])  + 

£U  {£U  +  £W'NMXW  ~  *M)+ 

(x(t)  -  i(t))'N(t)x(t)  -  x'(t)N(x(t)  -  *(<))  |  Z‘])  + 

£U  if”  [Jo^  +  u'(t)M(t)u(t)dt  |  Z‘]  )  .  (56) 

Using  the  orthogonality  property  of  the  Kalman  filter  [16],  namely  £[(x—x)Zx]  = 
0  where  Z  is  any  matrix,  it  immediately  follows  that 

£u  (tN)Xjx(tN)  +  j  x  (t)N(t)x(t)  +  ti'(f)M(J)tt(t)cft  | 

=  eu  (eu  \x  (tN)Xfx{tN)  +  J*”  x\t)N(t)x{t)  +  u\t)M{t)u(t)dt  I  z*]) 

+  Su  (e'  (tN)X}e(tN)  +  J*"  e  (t)Ne(t)dt^J  .  (57) 

The  result  now  follows  immediately. 
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Controller  Regions  No  noise 


Fig.  1.  Controller  decision  diagram  with  no  noise. 


Controller  Regions  with  noise 


Fig.  2.  Controller  decision  diagram  with  noise,  E[w]  =  0, £[«/«;]  =1.0 


Control  Input  Plant  States 
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States  vs  Time  t 
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Fig.  3.  Evolution  of  plant  states  with  time. 


Control  Input  vs  Time 
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Fig.  4.  Evolution  of  control  input  u(t)  with  time  for  the  switching  controller. 


